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Abstract 

Let  X  =  .bjb2b3...  be  a  random  variable  with  independent  binary 

digits  b^  taking  values  0  or  1  with  probabilities  p^  and  qn> 

When  does  X  have  a  density  function?  A  continuous  density  function? 

A  singular  distribution?  This  note  proves  that  the  distribution  of  X 

is  singular  if  and  only  if  the  tail  of  the  series  £[  logCp^/q^)  ]2 

diverges,  and  that  X  has  a  density  that  is  positive  on  some  interval 

if  and  only  if  logCp^/q^)  is  a  geometric  sequence  with  ratio  -j  for 

-  k 

n  greater  than  some  k,  and  in  that  case  the  fractional  part  of  2  X 
has  an  exponential  density  (increasing  or  decreasing  with  the  uniform 
density  a  special  case).  It  gives  a  sufficient  condition  for  X  to  have 
a  density,  (£log[2  max(pn  .q^)  ]  converges),  but  unless  the  tail  of  the 
sequence  log^^/q^)  is  geometric,  ratio  the  density  is  a  weird  one  that 
vanishes  at  least  once  in  every  interval. 


1. 


Introduction 


It  is  well  known  that  one  can  construct  a  uniform  random  variable 
by  choosing  the  binary  digits  with  successive  flips  of  a  good  coin, 

(p  =  y .  Such  considerations  date  back  to  the  beginnings  of  probability 
theory — indeed,  to  the  development  of  measure  and  integration  theory. 

For  bad  coins,  0<p<l,p^*st  the  resulting  number  has  a  dis¬ 
tribution  that  is  continuous  but  singular,  as  its  possible  values  form  a  set  of 
Lebesgue  measure  zero.  This  note  is  concerned  with  the  case  where  the 
binary  digits  are  independent,  but  not  identically  distributed.  Are  there 
any  interesting  random  variables  that  arise  from  this  situation?  It  turns 
out  that  there  are  some  interesting  singular  distributions,  and  that  there 
are  distributions  which  have  densities.  This  note  will  show  that  there  is 
essentially  one  conventional  type  density  that  can  arise  this  way — the 
exponential  (increasing  or  decreasing,  with  the  uniform  a  special  case), 
if  by  conventional  we  mean  a  density  that  is  positive  on  some  interval. 

We  will  also  find  necessary  and  sufficient  conditions  that  the  distribution 
be  singular.  The  conclusion  is  that  independent  binary  digits  lead  to  one 
of  four  possibilities: 

1)  A  singular  distribution.  This  happens  if  and  only  if  the  series 

oo 

2  log2 (p  /q  )  diverges  for  every  m.  A  special,  but  uninteresting, 
n-m 

case  Is  the  discrete  dibiuDuii...!  arising  when  p^q^  *  0  for  all 
suitably  large  n.  Some  interesting  continuous  but  singular 
distributions  arise.  See  Section  3. 

2)  A  piecewise-exponential  density,  with  pieces  equally  spaced  and 
of  similar  shape.  This  happens  only  when  the  tail  of  the  sequence 
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log(Pn/qn)  is  geometric  with  ratio  y.  It  is  the  only 
way  to  get  a  distribution  which  has  a  positive  derivative  on 
some  interval.  See  Section  2. 

3)  A  distribution  with  a  density,  but  a  strange  density  that  vanishes 

at  <ftf*^^tLnce  on  very  interval.  This  will  happen  if  Slog[2  maxCp^.q^)] 
converges  and  log(pn/q^)  is  not  geometric  with  ratio  y.  Section  4. 

4)  A  distribution  with  a  Lebesgue  decomposition  having  both  absolutely 
continuous  and  singular  parts.  We  conjecture  that  this  class  is 
empty,  but  have  not  been  able  to  rule  out  the  possibility  for  cases 
where  £log[2  maxtp^.q^)]  diverges  but  Slog2(pR/qn)  converges. 

2.  Distributions  with  Reasonable  Densities 

The  most  interesting  case  seems  to  be  the  assignment  of  probabilities 
to  the  bits  so  that  the  resulting  random  variable  has  a  conventional  density 
function.  We  will  show  that  there  is  essentially  only  one  way  to  do  this. 

The  general  statement  is  Theorem  2  belcw,  but  by  shifting  the  binary  decimal 
point  to  the  right  far  enough  we  may  assume  we  are  dealing  only  with  the 
fractional  part  taking  values  on  the  unit  interval,  and  in  that  case  we 
can  formulate  the  basic  requirement  as  follows: 

Theorem  1.  If  X  is  a  random  variable  on  the  unit  interval  with 
independent  binary  digits t 


X  -  .b  b  b  ... 
1  2  3 


b'a  independently  0  or  1,  and  if  the  distribution  function  of  X,  say 
F(x) ,  has  a  positive  derivative  at  ...  that  is,  F’ ( .  1) ,  F'(.Ol), 

F'(.OOl),  ...  all  exist  and  are  positive,  then  X  has  an  exponential 
distribution  with  density 


} 


f(x)  = 


O-  SX 


,  0  <  x  <  1 ,  -«'<3< 


ep  -  1 


and  the  probabilities  for  the  bits  of  X  are  given  by  the  formula 


P  [b±  =  0] 


P[bi  =  1] 


1  +  e 


1  +  e 


Proof:  Represent  the  probabilities  for  tne  pits  of  X  as  follows: 


P[bi  •=  0] 


1  +  e 


P[bi  -  1] 


1  +  e 


We  will  get  a  relation  between  the  c's  by  representing  F'(-l)  as  the 
limit  of  two  sequences  of  difference  quotients: 


F(.101)  -  F( ♦  1)  F(.1001)  -  F( .  1)  F(.  10001)  -  F(.l) 


F(.l)  -  F(,01)  F( . 1)  -  F(.Oll)  F( . 1)  -  F(.Olll) 


The  general  term  of  sequence  (1)  has  the  form 


1W) 


and  the  general  term  of  sequence  (2)  has  the  form 
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c„+c,+  . . .+c 

2  3  n 


Since  F'(.l)  exists  and  is  positive,  we  conclude  that  the  infinite 
product  converges,  that 


C2+C  ^  •  •  • 


F'(.l) 


Cj  c 

00  i .  00  4 

nW  rr(^) 


and  hence  Cj  ■  c2  +  c3  +  ...  . 

A  similar  argument  nn  F'(.Ol)  shows  that  c2  *  c3  +  c^  +  . . .  ; 
on  F'(.OOl)  shows  c3  ■  c4  +  C5  +  ...  and  thus  we  conclude  that  there 
is  a  6  such  that  c1  *  8/2,  c2  ■  8/4,  c3  *  8/8,  ...  .  This  provides 
the  formulas  for  the  bits  of  X  as  given  in  the  theorem.  We  still  must 
show  that  if  the  bits  of  X  are  given  by  those  probabilities,  then  X 
has  an  exponential  density  with  parameter  B.  (Note  that  B  can  be 
either  positive  or  negative,  with  B  ■  0  giving  the  uniform  density.) 

Writing 

bl  b2  b  3 

X  -  —  +  —  +  —  +  ... 

21  22  2 3 

we  express  the  characteristic  function  of  X  as  an  infinite  product 


B+it 


Using  the  relation 


1  -  eZ  -  (l+ez/2)(l-ez/2)  -  (l+ez/2)  (l+ez/4)  (l-ez/4)  -  ... 
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The  right  side  converges  to 


which,  with  z  =  3  +  it,  is  the  characteristic  function  of  X  with 
8x 

density  ye  on  0  <  x  <  1. 

To  get  a  more  general  theorem,  we  note  that  if  Y  has  a  positive  den¬ 
sity  on  some  interval  a  <  y  <  b  then  there  are  integers  r  and  k  such 
that  Y  has  a  positive  density  on  a  subinterval  of  the  form 


„  r  r  +  1  „  , 

a  -  T  <  y  <  — T~  5  b 
2  2 


and  hence  the  fractional  part  of 


has  a  density  or*  the  unit  interval; 


the  above  theorem  applies,  and  we  have: 


Theorem  2 .  If  Y  is  a  random  variable  with  independent  binary  digits , 
and  if  Y  has  a  distribution  which  has  a  positive  derivative  on  some 


interval ,  F'(y)  >0,  a  <  y  <  b,  then  Y  may  be  scaled  by  a  power  of  2, 

that  is y  its  binary  decimal  may  be  relocated t  so  that  its  fractional  part¬ 
is  exponentially  distributed  with  density 


0  <  x  <  1 


(For  some  6,  -<*=<$<«>,  with  the  uniform  density  corresponding  to 


**t*a*v*»i»w*~H 


b  =  o; 
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In  other  words  y  Y  may  be  represented  in  the  form  2  (MfX)  where 
M  is  a  random  integer ,  independent  of  the  fractional  part  X  having 
density  ye^x,  0  <  x  <  1. 

3.  Singular  Distributions 

Let  pn  *  P[bn  “  0]*  If  some  subsequence  of  the  p's  converges  to 
a  value  other  than  ,  then  the  distribution  of  X  *  .b1b2b3...  will  be 
singular,  for  its  possible  values  will  form  a  set  of  Lebesgue  measure 
0.  It  is  easy  to  get  many  representations  of  singular  variates  in  this 
way.  In  particular,  one  can  get  two  singular  distributions  whose 
convolution  has  an  exponential  or  a  uniform  distribution,  by  writing 

Xj  -  .0b20b40b60  ... 

X2  •  .bj0b30b50b7. . . 


where  the  b's  take  values  with  probabilities  given  by  Theorem  1. 

Then  Xj  +  X2  has  an  exponential  density  ye^x  on  0  <  x  <  1. 

To  get  two  singular  distributions  whose  convolution  is  the  ordinary 

-ax 

exponential  density  oie  ,  0  <  x  <  00 ,  write 


d3d2dld0*d-ld-2d-3 


where 


Plt^-0] 


_ 1 

1  +  e 


02^ 


Xj  ■  ...  d30  d1  0.d_1  0  d_3  0  d_g  .. 

X2  ■  . ,  .d4  0  d2  0  dQ .  0  d_2  0  d_4  0  . . . 


and 


We  now  turn  to  the  general  question  of  when  the  distribution  of 

X  =  .b1b2b3...  is  singular,  where  b^  takes  values  0  or  1  with 

probabilities  and  q^.  If  p^  does  not  converge  to  y  then  the 

distribution  of  X  will  be  singular,  but  what  happens  when  -*■  but 

not  according  to  the  formulas  of  Theorem  1?  We  know  that  X  cannot  have 

a  continuous  density  but  if  p^  **•  ^  very  very  quickly  we  might  expect 

that  X  will  have  some  kind  of  a  density,  though  a  weird  one.  The 

answer  is  yes,  there  is  always  a  density  if  pR  -*■  quickly  enough. 

Alternatively,  we  will  prove  that  the  distribution  of  X  is  singular 

00 

(including  discrete)  if  and  only  if  £  log2(p  /q^)  *  »  for  all  positive 

n«m  n 

integers  m. 

To  prove  this  result  we  need  a  preliminary  lemma  which  gives  a  formula 
for  F'(x)  when  it  exists: 

Lemma  1.  Let  X  -  .bjb2b3...  have  independent  binary  digits  with 

P[b  -0]  -  p  ,  P[b  «1]  ■  q  . 
n  rn  n  Mn 

Let  F  be  the  distribution  function  of  X.  If  F'  exists  at 
v  *  *viv2v3*“  then 


(3) 


F'(v)  -  [2gj(vj)  ]  [2g2  (v2) ][2g3(v3)  ]  ... 


where  g(0)  «  p  and  g  (1)  -  q  . 
n  n  n  n 

Proof:  Since  F'(v)  exists  it  can  be  represented 


F’(v)  - 


F(t)  -  F(s) 
t  -  8 


and  we  may  write 


lim 
8,t-*  v 
s<v<t 


-8- 

F' (.viv2V3* • •)  “  lim  2n[F(.v1v2. . .v  +2  n)  -  F(.VjV2...v  )]. 

n-*»  n  n 

Then  (3)  follows  from  the  fact  that  the  expression  in  brackets  is 
®l^vl^g2^v2)  •  *  ,gt/vn^  * 

Theorem  3.  Let  X  -  .b1b2b3...  have  independent  binary  digits  with 
bn  taking  values  0  or  1  with  probabilities  and  q^.  In  order 

that  X  have  a  singular  distribution  function  (derivative  equal  zero 
almost  everywhere)  it  is  necessary  and  sufficient  that  for  every  positive 
integer  m, 

n-m 

Proof:  Let  F  be  the  distribution  function.  It  has  a  finite  derivative 
almost  everywhere.  Thus  from  Lemma  1,  for  almost  all  x  ■  .  x^x^x^. . . 
we  have  F'  expressed  as  an  infinite  product: 

(4)  F'(.x1x2x3. . .)  -  UgjCx^  ][2g2(x2>  ]  [2g3(x3>  ]  ...  , 

where  8n(0)  ■  Pn  and  g^(l)  ■ 

Another  Interpretation  of  the  fact  that  F  has  a  derivative  almost 

everywhere  is  to  say  that  if  *viV2V3***  *8  c^osen  at  random  with  the 

v ' 8  independently  0  or  1  with  probability  then  with  probability  1 

the  product  in  (3)  converges  to  a  (possibly  zero)  constant.  According 

to  a  standard  theorem  on  infinite  products,  (see,  e.g.,  Knopp  [2],  p. 

223) ,  the  tail  of  the  product  in  (4)  converges  to  a  non-zero  constant  eL 

if  and  only  if  for  some  m, 

00 

2  1°g[2gn(xn) ]  -  L  . 
n»m 

Thus  the  question  of  whether  F'  is  positive  almost  everywhere  or  zero 
almost  everywhere  hinges  on  the  convergence  of  the  random  series 


2  Uog(p  /q  )  l2  -  00. 
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S  log[2gn(vn)  ], 
n=m 

where  the  v's  take  values  0  or  1  with  probability  y. 

We  apply  the  three  series  theorem  (see,  e.g.,  Fisz  [1],  p.  248)  after 
computing 

E{ log[2g  (v  ) ] }  =  ^  log(4p  q  ) ,  Variance  =  j[  log(p  /q  ) ]2 . 


If  the  tail  of  ^log^p^/q^)  converges,  so  does  the  tail  of  - 2l°g(4pncin) , 
for  the  terms  of  the  former  dominate  the  terms  of  the  latter.  (Write 
2pn  =  1  +  t,  2qn  *  1  -  t  then  note  that  log2 [(1+t) /(1-t)  ]  +  log(l-t2)  >  0 
for  -1  <  t  <  1.) 

Applying  the  three  series  theorem  to  the  random  series  Slog(2gn(vn) ) 
and  interpreting  the  result  in  terms  of  the  infinite  product  (4)  we 
conclude: 

oo 

If  ^  log2(p  /q  )  converges  for  some  m,  then  F'(x)  >  0  for 
n=m 

almost  all  x;  if  it  diverges  for  all  m,  then  F'(x)  -  0  for  almost 
all  x  and  hence  F  is  singular. 

4.  Absolute  Continuity 

To  recapitulate:  We  know  that  F  is  singular  if  and  only  if  the 
tail  of  Sl°g2(Pn/9n)  diverges,  and  hence  that  F  has  an  absolutely 
continuous  component  in  its  Lebesgue  decomposition  if  and  only  if  the 
tail  of  Slog2(pn/qn)  converges.  But  does  the  convergence  of  the  tail 
of  Slog2(pn/qn)  imply  that  F  is  purely  absolutely  continuous,  or 
can  it  have  both  an  a.c.  and  a  singular  part?  The  question  is  open,  but 
the  following  theorem  shows  that  if  pn  -*  y  fast  enough,  then  F  will 
have  no  singular  part. 


Theorem  A.  Let  X  *  .bjb2b3...  have  independent  binary  digits  with 
bR  taking  values  0  or  1  with  probabilities  pR  and  qR.  If  the 
series 

OO  00 

X  log[2  max(p  ,q  )]  -  £  log[l+(2p  -l|  ] 

i-1  1  1  i-1 

converges  then  the  distribution  of  X  is  absolutely  continuous. 

Proof:  Let 

00 

n~(2  max(p  ,q  )  ]  -  M, 
i-1  x 

where  0  <  M  <  ®  because  the  series  of  logarithms  converges.  We  will 
show  that  F  is  absolutely  continuous  by  showing  that  the  probability  of 
an  interval  is  less  than  M  times  the  length  of  the  interval: 

(5)  F(y)  -  F(x)  <  (y-x)M,  x  <  y. 


In  fact,  it  suffices  to  show  that  (5)  holds  if  x  and  y  are  in  the 
dense  set  of  terminating  binary  decimals,  and  in  that  case  we  may  write, 
with  g^O)  -  pt  and  g^l)  -  q^ 


FC.XjXj.  .  .xn+2  n)  -  F(.XjX2..  .xn)  -  g j (x j ) g2 (x£)  ... gjj(Xp) 

n 

£  2*"n]  [max(2p .  ,2q  ) 
i-1  x 

<  2-nM. 


A  Conjecture 


There  seems  to  be  one  question  that  remains,  which  we  put  in  the 


form  of  a  conjecture:  If  X  -  .bJb2b3...  has  independent  binary  digits , 
then  the  distribution  of  X  is  either  purely  discrete ,  purely  absolutely 
continuous ,  or  purely  continuous  and  singular. 


-li¬ 


lt  is  easy  to  show  that  it  is  purely  discrete  if  and  only  if  Pn<ln  * 

for  all  large  n,  and  it  is  purely  singular,  including  discrete,  if  and 

only  if  the  tail  of  Slog2(p  /q  )  diverges.  But  what  about  when  the 

n  n 

tail  converges,  but  not  fast  enough  for  ^log[2  maxCp^.q^)]  to  converge? 
The  resulting  F  has  a  positive  derivative  almost  everywhere,  but  can 
the  Lebesgue  decomposition  of  F  have  both  an  absolutely  continuous  and  a 
singular  part? 
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